AWS FargateにサイドカーパターンのFireLensコンテナも含めて一式作成するCloudFormationテンプレート

AWS FargateにサイドカーパターンのFireLensコンテナも含めて一式作成するCloudFormationテンプレート

#FireLens
#AWS Fargate
#AWS CloudFormation
#Fluent Bit
#CloudWatch Logs
大村 保貴

大村 保貴

facebook logohatena logotwitter logo
Clock Icon2021.09.05

この記事は公開されてから1年以上経過しています。情報が古い可能性がありますので、ご注意ください。

コンテナの実行基盤としてAWS Fargateを利用しています。FireLens（Fluent Bit）の動作確認のため手軽にクラスターごと作成・削除できる検証環境を欲していました。素のNginx（Webサーバ）にFireLens込みの実行環境をCloudFormationのテンプレートにまとめました。FireLensは最近検証していた内容を踏まえ補足説明します。

以下の環境を構築します。

本記事のCloudFormationのスタック作成にはrainコマンドを使用しています。

事前準備

VPC作成

Fargate作成のテンプレートではパブリックサブネットにELBを新規作成します。プライベートサブネットにNginxコンテナと、FireLensコンテナをデプロイします。既存VPCを流用しても問題ありません。 紹介するVPCテンプレートは以下のネットワーク構成を作成します。

検証環境のコスト考慮

検証環境ということもありVPCを維持するのにあたり、NAT Gatewayのランニングコストが気になります。NAT Gatewayの作成有無はテンプレートから切り替えできます。初期構築時に「Nat Gatewayを作るか、作らないか」という意図ではなく、NAT Gateway が必要なときに具体的にはプライベートサブネットからインターネットアクセスが必要なときはEnabeleNatGatewayをtrueにして、スタックの更新をかけます。NAT Gatewayを新規作成し、ルートテーブルを設定し直します。検証が終わったあとはfalseにして更新をかけるとNAT Gatewayを削除します。NAT Gatewayを維持するコスト削減を目的としています。NAT GatewayにアタッチするEIPも保持による課金を抑えるため、NAT Gatewayの削除と同時にEIPを解放します。

VPCテンプレート

折りたたみ 
---
AWSTemplateFormatVersion: "2010-09-09"
Description: Create network 3 layers with switching single Nat Gateway

Metadata:
  AWS::CloudFormation::Interface:
    ParameterGroups:
      - Label:
          default: Common Settings
        Parameters:
          - ProjectName
          - Environment
      - Label:
          default: VPC Settings
        Parameters:
          - VPCCidr
          - PublicSubnetCidr1
          - PublicSubnetCidr2
          - PrivateSubnetCidr1
          - PrivateSubnetCidr2
          - IsolatedSubnetCidr1
          - IsolatedSubnetCidr2
      - Label:
          default: NAT Gateway
        Parameters:
          - EnableNatGateway

Parameters:
  ProjectName:
    Description: Project Name
    Type: String
    Default: unnamed
  Environment:
    Description: Environment
    Type: String
    Default: dev
    AllowedValues:
      - prod
      - dev
      - stg
  VPCCidr:
    Description: VPC IP Range
    Type: String
    Default: 10.0.0.0/16
    AllowedPattern: '^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/([0-9]|[1-2][0-9]|3[0-2]))$'
  PublicSubnetCidr1:
    Description: Public Subnet 1 IP Range
    Type: String
    Default: 10.0.1.0/24
    AllowedPattern: '^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/([0-9]|[1-2][0-9]|3[0-2]))$'
  PublicSubnetCidr2:
    Description: Public Subnet 2 IP Range
    Type: String
    Default: 10.0.2.0/24
    AllowedPattern: '^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/([0-9]|[1-2][0-9]|3[0-2]))$'
  PrivateSubnetCidr1:
    Description: Private Subnet 1 IP Range
    Type: String
    Default: 10.0.17.0/24
    AllowedPattern: '^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/([0-9]|[1-2][0-9]|3[0-2]))$'
  PrivateSubnetCidr2:
    Description: Private Subnet 2 IP Range
    Type: String
    Default: 10.0.18.0/24
    AllowedPattern: '^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/([0-9]|[1-2][0-9]|3[0-2]))$'
  IsolatedSubnetCidr1:
    Description: Isolated Subnet 1 IP Range
    Type: String
    Default: 10.0.33.0/24
    AllowedPattern: '^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/([0-9]|[1-2][0-9]|3[0-2]))$'
  IsolatedSubnetCidr2:
    Description: Isolated Subnet 2 IP Range
    Type: String
    Default: 10.0.34.0/24
    AllowedPattern: '^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/([0-9]|[1-2][0-9]|3[0-2]))$'
  EnableNatGateway:
    Description: Enable NAT Gateway.
    Type: String
    Default: true
    AllowedValues: [true, false]

Conditions:
  EnableNatGateway: !Equals [true, !Ref EnableNatGateway]

Resources:
  # --------------------------------------------
  # VPC
  # --------------------------------------------
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref VPCCidr
      EnableDnsSupport: true
      EnableDnsHostnames: true
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-${Environment}-vpc
        - Key: Environment
          Value: !Sub ${Environment}

  # --------------------------------------------
  # Internet Gateway
  # --------------------------------------------
  # Create InternetGateway & VPC Attach
  InternetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-${Environment}-igw
        - Key: Environment
          Value: !Sub ${Environment}
  AttachGateway:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref VPC
      InternetGatewayId: !Ref InternetGateway

  # --------------------------------------------
  # NAT Gateway
  # --------------------------------------------
  # Create NatGateway
  NatGateway1:
    Type: AWS::EC2::NatGateway
    Condition: EnableNatGateway
    Properties:
      AllocationId: !GetAtt NatGatewayEIP1.AllocationId
      SubnetId: !Ref PublicSubnet1
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-${Environment}-natgw1
        - Key: Environment
          Value: !Sub ${Environment}
  NatGatewayEIP1:
    Type: AWS::EC2::EIP
    Condition: EnableNatGateway
    Properties:
      Domain: vpc
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-${Environment}-natgw-eip1
        - Key: Environment
          Value: !Sub ${Environment}

  # --------------------------------------------
  # Route Table
  # --------------------------------------------
  # Create Public RouteTable & Setting Routing
  PublicRouteTable1:
    Type: AWS::EC2::RouteTable
    DependsOn: AttachGateway
    Properties:
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-${Environment}-public-rtb1
        - Key: Environment
          Value: !Sub ${Environment}
  PublicRoute1:
    Type: AWS::EC2::Route
    DependsOn: AttachGateway
    Properties:
      RouteTableId: !Ref PublicRouteTable1
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref InternetGateway
  # Create Private RouteTable & Setting Routing
  PrivateRouteTable1:
    Type: AWS::EC2::RouteTable
    DependsOn: AttachGateway
    Properties:
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-${Environment}-private-rtb1
        - Key: Environment
          Value: !Sub ${Environment}
  PrivateRouteNatGW1:
    Type: AWS::EC2::Route
    Condition: EnableNatGateway
    DependsOn: AttachGateway
    Properties:
      RouteTableId: !Ref PrivateRouteTable1
      DestinationCidrBlock: 0.0.0.0/0
      NatGatewayId: !Ref NatGateway1
  # Create Isolated RouteTable & Setting Routing
  IsolatedRouteTable1:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-${Environment}-isolated-rtb1
        - Key: Environment
          Value: !Sub ${Environment}

  # --------------------------------------------
  # Public Subnet
  # --------------------------------------------
  # Public 1
  PublicSubnet1:
    Type: AWS::EC2::Subnet
    DependsOn: AttachGateway
    Properties:
      VpcId: !Ref VPC
      AvailabilityZone: !Select [0, !GetAZs ""]
      CidrBlock: !Ref PublicSubnetCidr1
      MapPublicIpOnLaunch: true
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-${Environment}-public-subnet1
        - Key: Environment
          Value: !Sub ${Environment}
  PublicSubnetRouteTableAssociation1:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PublicSubnet1
      RouteTableId: !Ref PublicRouteTable1
  # Public 2
  PublicSubnet2:
    Type: AWS::EC2::Subnet
    DependsOn: AttachGateway
    Properties:
      VpcId: !Ref VPC
      AvailabilityZone: !Select [1, !GetAZs ""]
      CidrBlock: !Ref PublicSubnetCidr2
      MapPublicIpOnLaunch: true
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-${Environment}-public-subnet2
        - Key: Environment
          Value: !Sub ${Environment}
  PublicSubnetRouteTableAssociation2:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PublicSubnet2
      RouteTableId: !Ref PublicRouteTable1

  # --------------------------------------------
  # Private Subnet
  # --------------------------------------------
  # Private 1
  PrivateSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      AvailabilityZone: !Select [0, !GetAZs ""]
      CidrBlock: !Ref PrivateSubnetCidr1
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-${Environment}-private-subnet1
  PrivateSubnetRouteTableAssociation1:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PrivateSubnet1
      RouteTableId: !Ref PrivateRouteTable1
  # Private 2
  PrivateSubnet2:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      AvailabilityZone: !Select [1, !GetAZs ""]
      CidrBlock: !Ref PrivateSubnetCidr2
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-${Environment}-private-subnet2
  PrivateSubnetRouteTableAssociation2:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PrivateSubnet2
      RouteTableId: !Ref PrivateRouteTable1

  # --------------------------------------------
  # Isolated Subnet
  # --------------------------------------------
  # Isolated 1
  IsolatedSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      AvailabilityZone: !Select [0, !GetAZs ""]
      CidrBlock: !Ref IsolatedSubnetCidr1
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-${Environment}-isolated-subnet1
        - Key: Environment
          Value: !Sub ${Environment}
  IsolatedSubnetRouteTableAssociation1:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref IsolatedSubnet1
      RouteTableId: !Ref IsolatedRouteTable1
  # Isolated 2
  IsolatedSubnet2:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      AvailabilityZone: !Select [1, !GetAZs ""]
      CidrBlock: !Ref IsolatedSubnetCidr2
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-${Environment}-isolated-subnet2
        - Key: Environment
          Value: !Sub ${Environment}
  IsolatedSubnetRouteTableAssociation2:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref IsolatedSubnet2
      RouteTableId: !Ref IsolatedRouteTable1

  # --------------------------------------------
  # Network ACL
  # --------------------------------------------
  # Public NACL
  PublicNetworkACL1:
    Type: AWS::EC2::NetworkAcl
    Properties:
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-${Environment}-public-nacl1
  NetworkACLEntryPublicIngress1:
    Type: AWS::EC2::NetworkAclEntry
    Properties:
      CidrBlock: "0.0.0.0/0"
      Egress: false
      NetworkAclId: !Ref PublicNetworkACL1
      Protocol: -1
      RuleAction: "allow"
      RuleNumber: 100
  NetworkACLEntryPublicEgress1:
    Type: "AWS::EC2::NetworkAclEntry"
    Properties:
      CidrBlock: "0.0.0.0/0"
      Egress: true
      NetworkAclId: !Ref PublicNetworkACL1
      Protocol: -1
      RuleAction: "allow"
      RuleNumber: 100
  # Private NACL
  PrivateNetworkACL1:
    Type: AWS::EC2::NetworkAcl
    Properties:
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-${Environment}-private-nacl1
  NetworkACLEntryPrivateIngress1:
    Type: AWS::EC2::NetworkAclEntry
    Properties:
      CidrBlock: "0.0.0.0/0"
      Egress: false
      NetworkAclId: !Ref PrivateNetworkACL1
      Protocol: -1
      RuleAction: "allow"
      RuleNumber: 100
  NetworkACLEntryPrivateEgress1:
    Type: "AWS::EC2::NetworkAclEntry"
    Properties:
      CidrBlock: "0.0.0.0/0"
      Egress: true
      NetworkAclId: !Ref PrivateNetworkACL1
      Protocol: -1
      RuleAction: "allow"
      RuleNumber: 100
  # Isolated NACL
  IsolatedNetworkACL1:
    Type: AWS::EC2::NetworkAcl
    Properties:
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-${Environment}-isolated-nacl1
  NetworkACLEntryIsolatedIngress1:
    Type: AWS::EC2::NetworkAclEntry
    Properties:
      CidrBlock: "0.0.0.0/0"
      Egress: false
      NetworkAclId: !Ref IsolatedNetworkACL1
      Protocol: -1
      RuleAction: "allow"
      RuleNumber: 100
  NetworkACLEntryIsolatedEgress1:
    Type: "AWS::EC2::NetworkAclEntry"
    Properties:
      CidrBlock: "0.0.0.0/0"
      Egress: true
      NetworkAclId: !Ref IsolatedNetworkACL1
      Protocol: -1
      RuleAction: "allow"
      RuleNumber: 100

  # NetworkACL Association
  PublicNetworkACLAssocation1:
    Type: AWS::EC2::SubnetNetworkAclAssociation
    Properties:
      SubnetId: !Ref PublicSubnet1
      NetworkAclId: !Ref PublicNetworkACL1
  PublicNetworkACLAssocation2:
    Type: AWS::EC2::SubnetNetworkAclAssociation
    Properties:
      SubnetId: !Ref PublicSubnet2
      NetworkAclId: !Ref PublicNetworkACL1
  PrivateNetworkACLAssocation1:
    Type: AWS::EC2::SubnetNetworkAclAssociation
    Properties:
      SubnetId: !Ref PrivateSubnet1
      NetworkAclId: !Ref PrivateNetworkACL1
  PrivateNetworkACLAssocation2:
    Type: AWS::EC2::SubnetNetworkAclAssociation
    Properties:
      SubnetId: !Ref PrivateSubnet2
      NetworkAclId: !Ref PrivateNetworkACL1
  IsolatedNetworkACLAssocation1:
    Type: AWS::EC2::SubnetNetworkAclAssociation
    Properties:
      SubnetId: !Ref IsolatedSubnet1
      NetworkAclId: !Ref IsolatedNetworkACL1
  IsolatedNetworkACLAssocation2:
    Type: AWS::EC2::SubnetNetworkAclAssociation
    Properties:
      SubnetId: !Ref IsolatedSubnet2
      NetworkAclId: !Ref IsolatedNetworkACL1

Outputs:
  ExportVPC:
    Value: !Ref VPC
    Export:
      Name: !Sub ${AWS::StackName}-VPC
  ExportPublicSubnet1:
    Value: !Ref PublicSubnet1
    Export:
      Name: !Sub ${AWS::StackName}-PublicSubnet1
  ExportPublicSubnet2:
    Value: !Ref PublicSubnet2
    Export:
      Name: !Sub ${AWS::StackName}-PublicSubnet2
  ExportPrivateSubnet1:
    Value: !Ref PrivateSubnet1
    Export:
      Name: !Sub ${AWS::StackName}-PrivateSubnet1
  ExportPrivateSubnet2:
    Value: !Ref PrivateSubnet2
    Export:
      Name: !Sub ${AWS::StackName}-PrivateSubnet2
  ExportPrivateRoutetable1:
    Value: !Ref PrivateRouteTable1
    Export:
      Name: !Sub ${AWS::StackName}-PrivateRoutetable1
  ExportIsolatedSubnet1:
    Value: !Ref IsolatedSubnet1
    Export:
      Name: !Sub ${AWS::StackName}-IsolatedSubnet1
  ExportIsolatedSubnet2:
    Value: !Ref IsolatedSubnet2
    Export:
      Name: !Sub ${AWS::StackName}-IsolatedSubnet2
  ExportIsolatedetable1:
    Value: !Ref IsolatedRouteTable1
    Export:
      Name: !Sub ${AWS::StackName}-Isolatedetable1

本検証環境は以下のVPCテンプレートをrainコマンドで以下のパラメータで作成した環境を用います。もちろんマネージドコンソール、AWS CLIから同様のCloudFormationのスタックを作成できます。

rain deploy ./vpc.yml sample-vpc-stack --params \
                    ProjectName=sample,\
                    Environment=dev,\
                    VPCCidr=10.0.0.0/16,\
                    PublicSubnetCidr1=10.0.1.0/24,\
                    PublicSubnetCidr2=10.0.2.0/24,\
                    PrivateSubnetCidr1=10.0.17.0/24,\
                    PrivateSubnetCidr2=10.0.18.0/24,\
                    IsolatedSubnetCidr1=10.0.33.0/24,\
                    IsolatedSubnetCidr2=10.0.34.0/24,\
                    EnableNatGateway=true

ECR作成

この後Fargateをデプロイするタスク定義の都合、FireLens（Fluent Bit）の独自設定ファイル込みイメージのを事前にアップロードする必要があります。本検証環境ではsample3-custom-logrouter-firelensリポジトリを作成しました。

FireLensイメージ作成

先ほど作成したリポジトリにアップロードするイメージを作成します。

カレントディレクトリにあるFluent Bitの設定ファイル（extra.conf）をイメージ内の/fluent-bit/etc/extra.confパスへ保存します。このパスをFargateのタスク定義内で指定する箇所があります。設定ファイルをコピーするパスを変更する場合は、後ほど紹介するFargateのテンプレート内のタスク定義の修正も合わせて実施してください。

FROM public.ecr.aws/aws-observability/aws-for-fluent-bit:2.19.0
COPY ./extra.conf /fluent-bit/etc/extra.conf

ディレクトリ構成は以下の状態です。

.
├── Dockerfile
└── extra.conf

Fluent Bitの設定ファイル（extra.conf）と、設定してある内容は以下です。

  • ELBのヘルスチェックによるアクセスログは除外
  • CloudWatch Logsにログを保存
  • CloudWatch Logsのログストリーム名はタスクごとに作成
[SERVICE]
    Flush 1
    Grace 30

[FILTER]
    Name grep
    Match webapp-firelens*
    Exclude log ^(?=.*ELB-HealthChecker\/2\.0).*$

[OUTPUT]
    Name   cloudwatch
    Match webapp-firelens*
    region ap-northeast-1
    log_group_name /$(ecs_cluster)
    log_stream_name App/$(ecs_task_id)
    auto_create_group true

設定内容については以下のリンクで説明しています。

イメージのアップロード

Fargateをデプロイするテンプレートのパラメータにイメージ名とタグ名を入力する項目があります。 本検証環境ではv1タグを付けて、手元でビルドしたイメージをアップロードします。

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 123456789012.dkr.ecr.us-east-1.amazonaws.com
docker build -t sample3-custom-logrouter-firelens:v1 .
docker tag sample3-custom-logrouter-firelens:v1 123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/sample3-custom-logrouter-firelens:v1
docker push 123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/sample3-custom-logrouter-firelens:v1

VPCと、FireLensのイメージをアップロードしたら準備は完了です。

Fargate作成

パブリックサブネットにELBを、プライベートサブネットに素のNginxコンテナ+FireLensをデプロイします。そのため以下の情報が必要になります。VPCを今回のテンプレートから作成した場合はOutputに書き出してある値をそのまま使います。

VPC関連

各種IDを入力

  • VPCID
  • PublicSubnet1
  • PublicSubnet2
  • PrivateSubnet1
  • PrivateSubnet2

FireLens

ECRにアップロードしたFireLensのイメージ名:タグを入力

  • ImageNameFirelens

以下の環境が作成できます。

Fargateテンプレート

Fluent Bitの設定ファイル（extraf.conf）のイメージ内保存先のパスを変更している場合は以下の箇所を適切なパスに変更してください。

          FirelensConfiguration:
            Type: "fluentbit"
            Options:
              config-file-type: "file"
              config-file-value: "/fluent-bit/etc/extra.conf"
折りたたみ 
AWSTemplateFormatVersion: "2010-09-09"
Description: Create Fargate*1 with B/G Deployment, AutoScaling and Firelens

Metadata:
  AWS::CloudFormation::Interface:
    ParameterGroups:
      - Label:
          default: Common Settings
        Parameters:
          - ProjectName
          - Environment
      - Label:
          default: ECS VPC Settings
        Parameters:
          - VPCID
          - PublicSubnet1
          - PublicSubnet2
          - PrivateSubnet1
          - PrivateSubnet2

Parameters:
  ProjectName:
    Description: Project Name
    Type: String
    Default: unnamed
  Environment:
    Description: Environment
    Type: String
    Default: dev
    AllowedValues:
      - prod
      - dev
      - stg
  VPCID:
    Type: AWS::EC2::VPC::Id
  PublicSubnet1:
    Description: "ELB Subnet 1st"
    Type: AWS::EC2::Subnet::Id
  PublicSubnet2:
    Description: "ELB Subnet 2nd"
    Type: AWS::EC2::Subnet::Id
  PrivateSubnet1:
    Description: "ECS Subnet 1st"
    Type: AWS::EC2::Subnet::Id
  PrivateSubnet2:
    Description: "ECS Subnet 2nd"
    Type: AWS::EC2::Subnet::Id
  DesiredCount:
    Type: Number
    Default: 1
  ClusterName:
    Type: String
    Default: cluster
  AppName:
    Type: String
    Default: webapp
  ServiceName:
    Type: String
    Default: service
  TaskDefinitionName:
    Type: String
    Default: taskdefinition
  ImageNameWebApp:
    Description: "Web Application Repository Name also Need to TagName"
    Type: String
    Default: "public.ecr.aws/nginx/nginx:latest"
  ImageNameFirelens:
    Description: "Firelens Repository Name also Need to TagName"
    Type: String
    Default: "public.ecr.aws/aws-observability/aws-for-fluent-bit:latest"

Resources:
  # --------------------------------------------
  # ELB
  # --------------------------------------------
  ELB1:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Type: "application"
      Name: !Sub ${ProjectName}-${Environment}-elb
      Scheme: "internet-facing"
      SecurityGroups:
        - !Ref SecurityGroup2
      Subnets:
        - !Ref PublicSubnet1
        - !Ref PublicSubnet2
      IpAddressType: "ipv4"
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-${Environment}-elb

  ELBListener1:
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      DefaultActions:
        - TargetGroupArn: !Ref TargetGroup1
          Type: "forward"
      LoadBalancerArn: !Ref ELB1
      Port: 80
      Protocol: "HTTP"
  ELBListener2:
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      DefaultActions:
        - TargetGroupArn: !Ref TargetGroup2
          Type: "forward"
      LoadBalancerArn: !Ref ELB1
      Port: 8080
      Protocol: "HTTP"

  TargetGroup1:
    Type: "AWS::ElasticLoadBalancingV2::TargetGroup"
    Properties:
      VpcId: !Ref VPCID
      Name: !Sub ${ProjectName}-${Environment}-tg1
      Protocol: "HTTP"
      HealthCheckPath: "/"
      Port: 80
      TargetType: ip
      HealthCheckIntervalSeconds: 10 # Default is 30.
      HealthyThresholdCount: 2 # Default is 5.
      HealthCheckTimeoutSeconds: 5
      UnhealthyThresholdCount: 2
      TargetGroupAttributes:
        - Key: "stickiness.enabled"
          Value: "false"
        - Key: deregistration_delay.timeout_seconds
          Value: "60" # default is 300.
        - Key: "stickiness.type"
          Value: "lb_cookie"
        - Key: "stickiness.lb_cookie.duration_seconds"
          Value: "86400"
        - Key: "slow_start.duration_seconds"
          Value: "0"
        - Key: "load_balancing.algorithm.type"
          Value: "round_robin"

  TargetGroup2:
    Type: "AWS::ElasticLoadBalancingV2::TargetGroup"
    Properties:
      VpcId: !Ref VPCID
      Name: !Sub ${ProjectName}-${Environment}-tg2
      Protocol: "HTTP"
      HealthCheckPath: "/"
      Port: 80
      TargetType: ip
      HealthCheckIntervalSeconds: 10 # Default is 30.
      HealthyThresholdCount: 2 # Default is 5.
      HealthCheckTimeoutSeconds: 5
      UnhealthyThresholdCount: 2
      TargetGroupAttributes:
        - Key: "stickiness.enabled"
          Value: "false"
        - Key: deregistration_delay.timeout_seconds
          Value: "60" # default is 300.
        - Key: "stickiness.type"
          Value: "lb_cookie"
        - Key: "stickiness.lb_cookie.duration_seconds"
          Value: "86400"
        - Key: "slow_start.duration_seconds"
          Value: "0"
        - Key: "load_balancing.algorithm.type"
          Value: "round_robin"

  # --------------------------------------------
  # CloudWatch Logs Group
  # --------------------------------------------
  # FireLens Stdout
  FireLensLogGroup:
    Type: "AWS::Logs::LogGroup"
    Properties:
      LogGroupName: !Sub "${ProjectName}-${Environment}-${ClusterName}-firelens-logs"
      RetentionInDays: 400

  # --------------------------------------------
  # ECS Fargate
  # --------------------------------------------
  # Cluster
  ECSCluster:
    Type: "AWS::ECS::Cluster"
    Properties:
      ClusterName: !Sub "${ProjectName}-${Environment}-${ClusterName}"
      ClusterSettings:
        - Name: containerInsights
          Value: enabled
      CapacityProviders:
        - "FARGATE_SPOT"
        - "FARGATE"

  # Service
  ECSService:
    Type: "AWS::ECS::Service"
    Properties:
      ServiceName: !Sub ${ProjectName}-${Environment}-${ServiceName}
      Cluster: !Ref ECSCluster
      LaunchType: "FARGATE"
      PlatformVersion: "1.4.0"
      DeploymentController:
        Type: CODE_DEPLOY
      DesiredCount: !Ref DesiredCount
      LoadBalancers:
        - TargetGroupArn: !Ref TargetGroup1
          ContainerName: !Ref AppName
          ContainerPort: 80
      NetworkConfiguration:
        AwsvpcConfiguration:
          AssignPublicIp: "DISABLED"
          SecurityGroups:
            - !Ref SecurityGroup1
          Subnets:
            - !Ref PrivateSubnet1
            - !Ref PrivateSubnet2
      TaskDefinition: !Ref ECSTaskDefinition
    DependsOn: ELBListener1
  # ECS TaskDefinition
  ECSTaskDefinition:
    Type: "AWS::ECS::TaskDefinition"
    Properties:
      Family: !Sub "${ProjectName}-${Environment}-${AppName}-${TaskDefinitionName}"
      TaskRoleArn: !GetAtt ECSTaskRole1.Arn
      ExecutionRoleArn: !GetAtt ECSTaskExecutionRole1.Arn
      NetworkMode: "awsvpc"
      RequiresCompatibilities:
        - "FARGATE"
      Cpu: "256"
      Memory: "512"
      ContainerDefinitions:
        - Essential: true
          Name: !Ref AppName
          Image: !Ref ImageNameWebApp
          LogConfiguration:
            LogDriver: "awsfirelens"
          PortMappings:
            - ContainerPort: 80
              HostPort: 80
              Protocol: "tcp"
        - Essential: true
          Name: "log_router"
          Image: !Ref ImageNameFirelens
          LogConfiguration:
            LogDriver: "awslogs"
            Options:
              awslogs-group: !Ref FireLensLogGroup
              awslogs-region: !Ref AWS::Region
              awslogs-stream-prefix: "FireLens"
          FirelensConfiguration:
            Type: "fluentbit"
            Options:
              config-file-type: "file"
              config-file-value: "/fluent-bit/etc/extra.conf"
          User: "0"

  # --------------------------------------------
  # Security Group
  # --------------------------------------------
  # Security Group for WebApp
  SecurityGroup1:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: !Sub ${ProjectName}-${Environment}-${AppName}-sg
      GroupDescription: Web App Security Group
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          SourceSecurityGroupId: !Ref SecurityGroup2
          Description: "Access from ELB"
      VpcId: !Ref VPCID
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-${Environment}-${AppName}-sg
  # Security Group for ELB
  SecurityGroup2:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: !Sub ${ProjectName}-${Environment}-elb-sg
      GroupDescription: ELB Security Group
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: "0.0.0.0/0"
          Description: "Access from Public / Blue"
        - IpProtocol: tcp
          FromPort: 8080
          ToPort: 8080
          CidrIp: "0.0.0.0/0"
          Description: "Access from Public / Green"
      VpcId: !Ref VPCID
      Tags:
        - Key: Name
          Value: !Sub ${ProjectName}-${Environment}-elb-sg

  # --------------------------------------------
  # IAM Role
  # --------------------------------------------
  # ECS Task Execution Role
  ECSTaskExecutionRole1:
    Type: "AWS::IAM::Role"
    Properties:
      RoleName: !Sub ${ProjectName}-${Environment}-${AppName}-ECSTaskExecutionRole
      Path: "/"
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service: ecs-tasks.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
  # ECS Task Role
  ECSTaskRole1:
    Type: "AWS::IAM::Role"
    Properties:
      Path: "/"
      RoleName: !Sub ${ProjectName}-${Environment}-${AppName}-ECSTaskRole
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service: ecs-tasks.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - !Ref ECSExecPolicy
        - !Ref SentCloudWatchLogsPolicy

  # --------------------------------------------
  # IAM Policy
  # --------------------------------------------
  # Allowed ECS Exec for Task Role
  ECSExecPolicy:
    Type: "AWS::IAM::ManagedPolicy"
    Properties:
      ManagedPolicyName: !Sub "${ProjectName}-${Environment}-ECSExecPolicy"
      Path: "/"
      PolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Action:
              - ssmmessages:CreateControlChannel
              - ssmmessages:CreateDataChannel
              - ssmmessages:OpenControlChannel
              - ssmmessages:OpenDataChannel
            Resource: "*"
  # Sent CloudWatch Logs for Task Role
  SentCloudWatchLogsPolicy:
    Type: "AWS::IAM::ManagedPolicy"
    Properties:
      ManagedPolicyName: !Sub "${ProjectName}-${Environment}-SentCloudWatchLogsPolicy"
      Path: "/"
      PolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Action:
              - logs:CreateLogStream
              - logs:CreateLogGroup
              - logs:DescribeLogStreams
              - logs:PutLogEvents
            Resource: !Sub arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:*

  # ------------------------------------------------------------#
  #  Auto Scaling Service
  # ------------------------------------------------------------#
  ServiceScalingTarget:
    Type: AWS::ApplicationAutoScaling::ScalableTarget
    Properties:
      MinCapacity: 1
      MaxCapacity: 4
      # ResourceIdの必要書式: service/クラスター名/サービス名
      ResourceId: !Sub service/${ECSCluster}/${ProjectName}-${Environment}-${ServiceName}
      RoleARN: !Sub "arn:aws:iam::${AWS::AccountId}:role/aws-service-role/ecs.application-autoscaling.amazonaws.com/AWSServiceRoleForApplicationAutoScaling_ECSService"
      ScalableDimension: ecs:service:DesiredCount
      ServiceNamespace: ecs
    DependsOn:
      - ECSService # ResourIdでサービスを参照するため先に作成されている必要がある

  ServiceScaleOutPolicy:
    Type: AWS::ApplicationAutoScaling::ScalingPolicy
    Properties:
      PolicyName: !Sub "${ProjectName}-${Environment}-${ServiceName}-ScaleOutPolicy"
      PolicyType: StepScaling
      ScalingTargetId: !Ref ServiceScalingTarget
      StepScalingPolicyConfiguration:
        AdjustmentType: ChangeInCapacity
        Cooldown: 60
        MetricAggregationType: Average
        StepAdjustments:
          - ScalingAdjustment: 1
            MetricIntervalLowerBound: 0

  ServiceScaleInPolicy:
    Type: AWS::ApplicationAutoScaling::ScalingPolicy
    Properties:
      PolicyName: !Sub "${ProjectName}-${Environment}-${ServiceName}-ScaleInPolicy"
      PolicyType: StepScaling
      ScalingTargetId: !Ref ServiceScalingTarget
      StepScalingPolicyConfiguration:
        AdjustmentType: ChangeInCapacity
        Cooldown: 60
        MetricAggregationType: Average
        StepAdjustments:
          - ScalingAdjustment: -1
            MetricIntervalUpperBound: 0

  # CloudWatch Alarms
  ServiceScaleOutAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmName: !Sub "${ProjectName}-${Environment}-${ServiceName}-ScaleOutAlarm"
      EvaluationPeriods: 1
      Statistic: Average
      TreatMissingData: notBreaching
      Threshold: 10
      AlarmDescription: Alarm to add capacity if CPU is high
      Period: 60
      AlarmActions:
        - !Ref ServiceScaleOutPolicy
      Namespace: AWS/ECS
      Dimensions:
        - Name: ClusterName
          Value: !Ref ECSCluster
        - Name: ServiceName
          Value: !Sub "${ProjectName}-${Environment}-${ServiceName}"
      ComparisonOperator: GreaterThanThreshold
      MetricName: CPUUtilization

  ServiceScaleInAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmName: !Sub "${ProjectName}-${Environment}-${ServiceName}-ScaleInAlarm"
      EvaluationPeriods: 1
      Statistic: Average
      TreatMissingData: notBreaching
      Threshold: 5
      AlarmDescription: Alarm to reduce capacity if container CPU is low
      Period: 300
      AlarmActions:
        - !Ref ServiceScaleInPolicy
      Namespace: AWS/ECS
      Dimensions:
        - Name: ClusterName
          Value: !Ref ECSCluster
        - Name: ServiceName
          Value: !Sub "${ProjectName}-${Environment}-${ServiceName}"
      ComparisonOperator: LessThanThreshold
      MetricName: CPUUtilization

本検証環境は以下のFargateテンプレートをrainコマンドで以下のパラメータで作成した環境を用います。

rain deploy ./fargate.yml sample3-fargate-stack --params \
	ProjectName=sample3,\
	Environment=dev,\
	ClusterName=cluster,\
	ServiceName=service,\
	TaskDefinitionName=taskdefinition,\
	AppName=webapp,\
	DesiredCount=1,\
	ImageNameWebApp=public.ecr.aws/nginx/nginx:latest,\
	ImageNameFirelens=123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/sample3-custom-logrouter-firelens:v1,\
	VPCID=vpc-0f6d43eea7f6e7fdf,\
	PublicSubnet1=subnet-059f8f126d382354b,\
	PublicSubnet2=subnet-0ac4ece5ea47938a6,\
	PrivateSubnet1=subnet-08a24e56d05376997,\
	PrivateSubnet2=subnet-0f625692b77783cec

10分ほどでFargateの構築、コンテナの起動が完了します。

補足

起動してきた素のNginxコンテナにはAutoScalingの設定が加えてあります。当初Fluent Bitの設定を検証していたとき、AutoScalingは不要だろうと思い設定していませんでした。Fluent Bitのプラグインを検証してるとAutoScalingしてかつ、ログ流量が多くないと気が付かなかったログストリームのクォータの問題がありました。そのため、AutoScaling設定と、Fluent Bitの設定はその対策を施してあるものを載せています。

詳しくは以下のリンクをご参照ください。

コンテナの確認

タスク確認

Nginxコンテナ（webaap）と、自前の設定ファイル込みのFireLensコンテナ（log_router）が起動しています。

WebブラウザからELBにアクセスするとNginxのデフォルトページを確認できます。

CloudWatch Logsの確認

Nginxコンテナのログ

FireLensのFluent Bit経由でCloudWatch Logsへ保存されます。ロググループ、ログストリームはFluent Bitの自前の設定ファイルに書いた内容で作成されています。ログストリームの末尾にはタスクIDが付与され、タスクごとにログストリームも分けることができます。

Nginxコンテナのアクセスログは以下の内容が記録されていました。

{
    "container_id": "c55844d79014427fad38dddb7ca529df-1393473712",
    "container_name": "webapp",
    "ecs_cluster": "sample3-dev-cluster",
    "ecs_task_arn": "arn:aws:ecs:ap-northeast-1:123456789012:task/sample3-dev-cluster/c55844d79014427fad38dddb7ca529df",
    "ecs_task_definition": "sample3-dev-webapp-taskdefinition:11",
    "log": "10.0.1.142 - - [05/Sep/2021:05:42:46 +0000] \"GET / HTTP/1.1\" 200 612 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36\" \"11.22.33.44\"",
    "source": "stdout"
}

FireLensのログ

Fluent Bitは経由せず標準のログドライバー(awslogs)からCloudWatch Logsへ保存されます。ロググループと、ログストリームのタスクIDより前半部分はFargateのタスク定義で設定した内容です。こちらのログストリーム末尾のタスクIDはデフォルトで付与されます。

詳しくは以下をご参照ください。

保存されていたログの内容はFluent Bit起動時のログしかありませんでした。こちらはFireLensコンテナのデバッグ目的のログ保存です。たとえばFluent BitからCloudWatch Logsへ送信時にスロットリングが発生している場合はこちらにログが記録されます。

再掲になりますが、以下のリンクはスロットリング発生時、FireLensコンテナに記録されたログを載せております。

CloudWatch Logsへログ送信する権限不足（IAMロースの設定ミス）のログもFireLensコンテナに記録されます。

おわりに

CloudFormationのテンプレートの紹介とともに、最近のFireLens検証から得た知見をあわせて紹介させていただきました。いろいろ検証して時間がかりました。今から勉強される方はショートカットできるように、どなたかのお役に立てれば幸いです。

Share this article

facebook logohatena logotwitter logo

関連記事

Fluent Bit でデバッグログを有効にする方法を教えてください

Fluent Bit でデバッグログを有効にする方法を教えてください

User avatar
hato
2023.10.19
ECS FargateでFireLens(AWS for Fluent Bit)の設定ファイルをS3から参照してみた

ECS FargateでFireLens(AWS for Fluent Bit)の設定ファイルをS3から参照してみた

User avatar
新澤忠士
2023.07.26
Firelensで起動するコンテナの環境変数を利用して起動したリージョンのCloudWatch Logsにログ出力する

Firelensで起動するコンテナの環境変数を利用して起動したリージョンのCloudWatch Logsにログ出力する

User avatar
岩城 匠朗
2023.04.28
FireLens(Fluent Bit)におけるタグの仕組みについて調査してみた

FireLens(Fluent Bit)におけるタグの仕組みについて調査してみた

User avatar
つくぼし
2023.03.21
Classmethod's white logo
Facebook's logo
X's logo
YouTube's logo

© Classmethod, Inc. All rights reserved.